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PREFACE 
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would  like  to  thank  the  Armstrong  Laboratory’s  Human  Resources  Directorate  (AL/HRD)  and 
in  particular  the  Technical  Training  Research  Division  for  providing  an  environment 
supportive  of  this  work.  Several  AIVHRD  individuals  were  of  specific  help;  Colonel  Rodger 
Ballentine  and  Drs.  Scott  Newcomb,  Hendrick  Ruck,  and  Wesley  Regian.  To  the  remainder 
of  the  Division  and  to  the  Library  staff,  thanks  for  all  of  your  assistance. 
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THE  EFFECT  OF  INSTRUCTOR-STUDENT  INTERACTION  ON  ACHIEVEMENT 
IN  COMPUTER-BASED  TRAINING  (CBT) 

SUMMARY 


The  role  of  the  instructor  in  computer  based  training  (CBT)  has  not  been  studied.  However, 
the  role  of  the  instructor  in  traditional  instruction  (Tl)  has  been  studied  and  has  been  shown 
to  influence  student  achievement.  One  of  the  key  findings  from  the  Tl  research  is  that 
instructor-student  interaction  is  positively  related  to  achievement.  The  present  investigation 
varied  instructor-student  interaction  (present/absent)  in  a  CBT  setting.  Subjects  worked  a 
spreadsheet  tutorial  and  then  were  asked  to  use  the  spreadsheet  to  compute  statistical  values. 
The  results  showed  that  both  interaction  and  no-interaction  Ss  equally  understood  the  spreadsheet 
commands  but  that  those  Ss  who  received  instructor  interactions  scored  higher  on  actually 
using  the  commands  to  compute  statistical  values.  Low-ability  Ss  appeared  to  benefit  most 
from  instructor  interaction.  The  results  were  discussed  both  in  terms  of  past  research  and  in 
terms  of  helping  to  define  the  role  of  the  CBT  instructor. 


I.  INTRODUCTION 

Computer-based  training  (CBT)  research  has  typically  focused  on  comparing  a  CBT  course 
with  a  corresponding  traditional  instruction  (Tl)  course.  Compared  to  a  similar  Tl  course,  CBT 
generally,  but  not  always,  produces  increases  in  learning  and  retention  while  concurrently 

requiring  less  time  than  Tl  (Fletcher  &  Rockway,  1986,  Goodwin,  Goodwin,  Nansel,  &  Helms, 

1986;  Kulik  &  Kulik,  1986,  1987;  McCombs,  Back,  &  West,  1984;  O'Neil,  1986).  However, 
CBT  results  have  not  always  been  positive ,  there  are  instances  in  which  CBT  did  not  produce 
increases  in  performance  or  decreases  in  learning  time  (Goodwin  et  al.,  1986;  McCombs  et 
al„  1984). 

In  general,  there  has  been  little  research  on  maximizing  performance  within  a  CBT  system 
(Gillingham  &  Guthrie,  1987).  Conversely,  there  is  a  long  history  of  research  on  variables 

that  influence  achievement  in  Tl  systems.  One  of  the  most  researched  variables  is  instructor 

behavior.  Tl  research  has  produced  a  relatively  high  degree  of  consensus  as  to  whai  an 

effective  instructor  does  versus  what  a  not-so-effective  instructor  does,  with  "effective'’  being 
defined  in  terms  of  academic  achievement  (Brophy,  1986;  Brophy  &  Good,  1986;  Rosenshine, 
1983)  Yet  CBT  research  has  neglected  the  role  of  the  instructor  (Moore,  1988).  Little  is 

known  about  whether  or  not  Tl  instructor  variables  transfer  to  CBT. 

In  a  study  that  did  examine  the  role  of  the  CBT  instructor,  Moore  (1988)  found  that  students 
who  had  teachers  with  positive  attitudes  scored  higher  than  those  in  classes  with  teachers 

with  negative  attitudes.  In  a  review  of  CBT  studies,  McCombs  et  al.  (1984)  found  that  two 
factors  were  critical  to  the  success  of  the  CBT  courses:  (a)  adequate  opportunities  for 
student-instructor  interactions,  and  (b)  the  incorporation  of  group  activities  with  individualized 
training. 

The  instructor-student  interaction  requirement  noted  by  McCombs  et  al  (1984)  is  a  significant 
finding  in  that  one  of  the  most  consistently  reported  positive  Tl  instructor  behaviors  is  frequent 
but  short  instructor- student  interactions;  i.e. ,  an  increase  in  instructor-student  interactions  produces 
an  increase  in  achievement  (Brophy,  1986;  Brophy  &  Good,  1986;  Rosenshine,  1983).  Therefore, 
a  Tl  instructor  behavior  which  may  transfer  to  CBT  is  instructor-student  interaction. 
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The  purpose  of  the  present  effort  was  to  examine  the  effect  of  instructor-student  interaction 
in  CBT.  Based  on  the  Tl  instructor  literature,  it  was  hypothesized  that  increased  instructor-student 
interaction  would  produce  increased  achievement. 


II.  METHOD 


Subjects 

The  subjects  were  25  (15  female  and  10  male)  college  juniors  and  seniors  enrolled  in  a 
Business  Statistics  class.  As  part  of  a  project  designed  to  teach  students  how  to  use  computer 
spreadsheet  software  to  perform  statistical  computations,  Ss  volunteered  to  participate  in  a 
spreadsheet  tutorial  for  extra  credit.  The  extra  credit  was  awarded  for  project  completion,  not 
for  project  performance.  All  Ss  completed  a  survey  to  assess  their  previous  personal  computer 
(PC)  experience. 


Experimental  Materials 

The  spreadsheet  tutorial  was  part  of  a  larger  commercial  software  tutorial  package  designed 
for  an  integrated  spreadsheet-word  processing-database  program.  The  tutorial  is  linear  and 
learner-controlled;  however,  Ss  do  have  the  capability  to  repeat  a  lesson  if  desired. 

For  the  purposes  of  this  study,  the  larger  tutorial  was  modified  to  include  only  the  introduction 
to  the  integrated  package  plus  that  portion  of  the  tutorial  software  devoted  to  the  use  of  the 
spreadsheet.  The  introduction  portion  (Part  A)  contained  four  lessons,  and  the  spreadsheet 
portion  (Part  B)  contained  eight  lessons.  The  tutorials  were  run  on  Tandy  1000SX  PCs. 

An  exercise  designed  to  evaluate  mastery  of  the  spreadsheet  tutorial  commands  was  added 
to  the  experimental  software.  Because  the  Ss  were  volunteers  from  a  Business  Statistics 
class,  the  exercise  used  simple  statistical  calculations  as  the  vehicle  for  evaluating  spreadsheet 
mastery.  Consequently,  the  experimental  material  consisted  of  a  CBT  spreadsheet  tutorial 
modified  to  include  a  statistics-based  exercise.  The  statistics  exercise  was  also  run  on  a  PC. 


Procedure 

Ss  were  randomly  assigned  by  spreadsheet/PC  experience  to  one  of  two  instructor-student 
interaction  modes.  Group  I  (n=13)  had  no  instructor-initiated  interactions.  All  Group  I  interactions 

were  initiated  by  the  student  and  consisted  of  requests  by  the  students  for  help  in  overcoming 

an  obstacle  in  the  tutorial.  Group  II  (n=12)  experienced  the  same  type  of  student-initiated 
interactions  as  those  experienced  by  Group  I.  In  addition.  Group  II  was  exposed  to  multiple 
instructor-initiated  interactions.  Ss  worked  individually  on  both  the  tutorial  and  the  statistics 
exercise. 

Both  groups  worked  the  CBT  tutorial  in  three  sessions.  In  session  one,  all  Ss  started  on 

lesson  A1  and  worked  in  the  tutorial  for  90  minutes.  In  the  second  session,  all  Ss  started 

on  lesson  B1  and  worked  through  the  last  lesson,  B8.  In  the  third  session,  all  Ss  started 

on  lesson  B3  and  again  worked  through  the  last  lesson,  lesson  B8.  Consequently,  all  Ss  had 

a  single  exposure  to  lessons  A1  though  A4  and  repeated  exposure  to  lessons  B1  through  B8 
Because  each  S  went  at  his/her  own  speed,  Ss'  total  time  on  task  varied.  At  the  completion 

of  lesson  B8  on  day  3,  all  Ss  were  given  the  statistics  exercise  designed  to  evaluate  their 

mastery  of  the  tutorial  material.  Ss  had  30  minutes  to  work  on  the  exercise. 


During  the  start-up  period  of  the  project  (i.e.,  the  first  15  minutes  of  the  first  session),  the 
instructor  responded  to  all  questions  in  both  groups  to  ensure  that  the  Ss  were  properly  logged 
into  the  tutorial.  For  both  groups,  the  instructor  also  responded  to  all  student-initiated  interactions 
with  one  or  more  of  three  responses:  (a)  "Try  pushing  the  [ESCAPE]  key”;  (b)  “Try  pushing 
the  [SPACE]  bar”;  or  (c)  “Re  boot  the  system  and  start  over.”  These  suggestions  were  given 
in  sequence  (c.g.,  if  “Try  pushing  the  [ESCAPE]  key”  did  not  work,  the  S  was  told  to  "Try 
pushing  the  [SPACE]  bar.')  For  Group  I  Ss,  these  suggestions  were  the  onlv  instructor 
interactions  experienced  after  the  first  15  minutes  of  session  one. 

In  addition  to  the  interactions  listed  above,  Group  II  Ss  also  experienced  instructor-initiated 
interactions.  In  the  first  session,  the  instructor  initiated  four  interactions  with  each  S.  In 
sessions  two  and  three,  the  instructor  initiated  three  interactions  and  one  interaction,  respectively. 
These  interactions  were  related  to  the  location  of  keys  on  the  Tandy  keyboard  (e.g.,  shortly 
before  needing  to  use  the  Back  Slash  (\)  key,  the  instructor  would  tell  the  Group  II  Ss  where 
that  key  was  located.)  Key  location  was  explained  and  diagrammed  in  instructions  given  to 
all  Ss;  but  for  most  Ss,  key  location  on  the  Tandy  keyboard  was  a  minor  problem  due  to 
previous  exposure  to  an  IBM  keyboard.  Instructor-initiated  interactions  lasted  between  5  and 
10  seconds. 

It  should  be  noted  that  in  no  instance  did  the  instructor  provide  information  that  was  not 
available  to  the  Ss  elsewhere.  Also,  in  no  instance  did  the  instructor  comment,  provide 
feedback,  or  give  praise  on  the  Ss'  performances  on  the  tutorial. 

Dependent  Measures 

Two  dependent  measures  were  recorded.  First,  the  Ss'  performance  on  the  exercise  was 
scored.  Second,  Ss  also  recorded  which  spreadsheet  commands  they  used.  Because  most 
procedures  can  be  performed  .n  more  than  one  way  (e.g.,  a  cell  entry  can  be  changed  via 
an  EDIT  command  or  by  simply  retyping  the  entry),  this  second  measure  was  recorded  to 
assess  how  many  different  commands  were  actually  used  during  the  exercise. 

III.  RESULTS 

Means  and  standard  deviations  for  Spreadsheet  Performance  and  Use  of  Spreadsheet 
Commands  are  given  in  Table  1.  Due  to  the  small  sample  sizes  (and  possible  problems  with 
the  assumption  of  normality),  the  Mann-Whitney  U  non-parametric  test  statistic  was  used  to 
analyze  differences  between  Group  I  (no  instructor  initiated  interaction)  Ss  and  Group  II 
(instructor-initiated  interaction)  Ss. 

Table  1.  Spreadsheet  Performance  and  Use  of  Spreadsheet  Commands: 

Means  and  Standard  Deviations 


Spreadsheet  Performance 
Mean  SD 


Group  1 

(No  Interaction) 

58.000 

18.257 

Group  II 

(Interaction) 

72.417 

7.403 

Use  of  Spreadsheet 

Commands 

Group  1 

(No  Interaction) 

32.308 

7.250 

Group  II 

(Interaction) 

30.833 

8.483 

Exercise  Performance 


Group  II  (instructor-initiated  interaction)  Ss  significantly  outperformed  Group  I  (no  instructor- 
initiated  interaction)  Ss  (Mann-Whitney  U  =  34.50,  £  <  .017). 

Use  of  Spreadsheet  Commands 


There  was  no  difference  in  command  usage  between  Group  I  Ss  and  Group  II  Ss; 
(Mann-Whitney  U  =  82.00,  £  <  .824). 

Sex  Differences 


Performance  differences  between  male  and  female  Ss  were  not  significant  (for  Spreadsheet 
Performance,  Mann-Whitney  U  =  56.00,  p  <.289;  for  Use  of  Spreadsheet  Commands, 
Mann-Whitney  U  =  69.50,  £  <  .755). 


IV.  DISCUSSION 

The  hypothesis  that  increased  instructor-student  interaction  would  lead  to  increased 
achievement  was  supported.  Given  the  limited  length  of  the  CBT  program  used  in  this 
experiment,  the  degree  of  difference  of  increased  achievement  between  the  two  groups  was 
surprising.  For  some  reason,  having  the  instructor  interact  with/take  notice  of/care  about  a 
student  affected  the  student  to  the  point  where  it  increased  his/her  achievement.  The  underlying 
cause  for  the  difference  in  achievement  did  not  seem  to  be  knowledge;  all  Ss  seemed  to 
equally  use  the  commands  presented  in  tne  tutorial.  The  difference  was  in  how  well  the 
commands  were  used. 

Nor  was  the  difference  in  achievement  due  to  praise  or  feedback,  neither  of  which  was 
given  by  the  instructor.  Unless  relatively  brief  human  interaction  is  defined  as  praise,  praise 
was  not  a  factor  in  this  study.  Extra  credit  for  higher  performance  on  the  exercise  also  was 
not  a  factor,  for  all  Ss  received  the  same  amount  of  extra  credit  regardless  of  their  performance. 

A  clue  as  to  why  Group  I  Ss  did  not  perform  as  well  as  Group  II  Ss  comes  from 
observations  made  by  the  CBT  instructor.  It  seemed  that  Group  I  Ss  used  the  space  bar 
more  frequently  than  did  Group  li  Ss.  In  this  study's  tutorial,  Ss  had  the  capability  to  literally 
space-bar  their  way  through  the  tutorial.  That  is,  rather  than  actually  performing  the  requested 
tutorial  action,  Ss  could  depress  the  space  bar  and  step  through  the  program.  Although  not 
measured,  Group  I  Ss  (no  interaction)  seemed  to  take  this  approach  more  frequently. 

Consequently,  although  both  groups  were  equally  exposed  to  the  material,  Group  II  Ss  (interaction) 
seemed  to  actually  perform  the  steps  of  the  tutorial  more  often.  If  in  fact  Group  II  Ss  did 
spend  more  time-on-task,  the  space  bar  behavior  could  account  for  the  difference  in  achievement. 
The  difference  in  standard  deviation  between  the  two  groups  could  also  be  a  result  of  the 
differing  amounts  of  actual  time-on-task. 

Although  the  small  number  of  cases  in  this  study  prevents  anything  more  than  a  reporting 
of  the  following,  it  was  noted  that  there  appeared  to  be  an  interaction  between  group 

(interaction/no  interaction)  and  spreadsheet  experience  level  (high/low).  In  general,  Ss  with 

previous  experience  in  using  a  spreadsheet  performed  roughly  the  same  across  instructor 
interaction  levels.  However,  low-experience  Ss  who  interacted  with  the  instructor  scored  higher 
than  did  low-experience  Ss  who  did  not  interact  with  the  instructor.  Also, 

low-experience/no- interaction  Ss  appeared  to  be  the  students  who  more  frequently  used  the 
space  bar  to  sequence  through  the  tutorial  and  consequently  spent  less  time-on-task. 
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This  observation  is  generally  in  keeping  with  other  research  which  has  suggested  that 
high-skill-level  students  benefit  more  from  CBT  than  do  low-skill  students,  at  least  in  moderate 
to  high  complexity  tasks  (Adams,  Waldrop,  Justen,  &  McCrosky,  1987,  Hativa  &  Shorer,  1989; 
Klein  &  Keller,  1990;  Whitney  &  Urquhart,  1990).  High-skill  students  may  be  in  less  need  of 
teacher  support,  Regardless  of  teacher  interaction,  these  students  are  capable  of  doing  the 
tutorial  and  do  so  without  need  of  monitoring  or  encouragement.  On  the  other  hand,  low-skill 
students  may  have  greater  difficulty  interacting  with  the  computer  and/or  the  tutorial  and 
therefore  may  be  the  ones  who  require  increased  human  interaction-  both  to  get  started  and 
to  stay  on  task. 

If  the  explanations  offered  above  are  accurate,  they  suggest  that  brief  human  interaction 
serves  to  keep  students  on  task  more  so  than  no  human  interaction.  Also,  if  I  am  a 
low-skill/experience  student  and  either  do  not  know  what  to  do  or  encounter  a  problem  in  the 
CBT  courseware,  I  will  probably  not  seek  assistance.  Instead,  I  will  try  to  get  through  the 
exercise  as  quietly  as  possible  (eg.,  by  using  the  space  bar).  However,  if  I  know  that  a 
teacher  is  going  to  be  initiating  interactions  with  me  and  is  therefore  going  to  be  aware  of 
how  well  or  how  poorly  I  am  doing,  then  I  may  try  to  stay  on  task  more. 

Due  to  the  manner  in  which  the  Group  II  interactions  occurred,  instructor  monitoring  of  the 
students  was  confounded  with  interaction.  For  the  instructor  to  know  when  to  interact  with 
an  appropriate  comment,  the  instructor  had  to  know  when  a  student  was  approaching  a  particular 
point  in  the  tutorial.  In  order  to  know  this,  the  instructor  had  to  constantly  monitor  the  students' 
progress.  Consequently,  while  the  Group  I  instructor  sat  at  a  desk  and  waited  for  students 
to  request  assistance,  the  Group  II  instructor  was  constantly  walking  around  the  room  and 
visually  checking  on  where  Ss  were  in  the  tutorial.  Therefore,  it  may  be  that  monitoring,  and 
not  interaction,  was  the  basis  for  Group  ll  s  higher  achievement. 

These  results  add  to  the  results  reported  by  Moore  (1988),  who  found  that  CBT  instructors 
with  positive  attitudes  produced  higher  achievement  than  did  CBT  instructors  with  negative 
attitudes.  Evidently,  instructor  interaction  can  also  affect  achievement.  Whether  or  not  the 
interaction  needs  to  be  tied  to  course  content  is  unknown.  It  may  be  that  CBT  instructors 
should  interact  with  students  in  order  to  maximize  achievement,  but  the  interactions  may  not 
need  to  be  related  to  the  material  being  covered. 


V.  IMPLICATIONS 

The  relatively  short-term  nature  of  the  tutorial  used  in  this  experiment  obviously  limits  the 
generalization  of  the  present  results.  That  limitation  not  withstanding,  the  specific  conclusion 
obtained  is  that  brief  instructor-initiated  interactions  can  increase  achievement  in  CBT.  However, 
instructor  monitoring  without  interaction  may  produce  the  same  result. 

It  may  also  be  true  that  the  CBT  instructor  can  most  influence  group  achievement  most 
by  spending  relatively  more  time  with  low-skill  students.  Skill  could  be  defined  by,  for  example, 
selection  scores  (Scholastic  Aptitude  Test,  Grade  Point  Average,  proficiency  test,  etc.)  or  by 
initial  student  performance  on  the  CBT  program.  Therefore,  CBT  instructors  need  to  be  able 
to  identify  those  students  with  low  ability/aptitude  so  that  relatively  more  time  can  be  allocated 
to  them  early  in  the  course. 

Because  the  role  of  the  instructor  in  CBT  is  frequently  undefined,  the  present  results  give 
some  direction  as  to  what  a  CBT  instructor  can  do  to  influence  achievement.  Moreover, 
because  instructor-initiated  interactions  are  controlled  by  the  instructor,  these  interactions  should 
also  be  built  into  the  larger  learning  system.  For  example,  in  addition  to  being  included  in 
the  CBT  courseware,  they  should  also  become  part  of  the  instructor  evaluation  system. 
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The  major  implication  from  the  present  investigation  is  that  instructor  interaction  does  seem 
to  influence  achiement  in  CBT.  The  results  obviously  support  Moore’s  research  (1988)  and 
McCombs  (1984)  suggestions.  There  is  simply  something  about  having  another  human  around 
and  aware  of  your  actions  that  alters  your  behavior.  Even  in  the  best-designed,  best-built, 
and  best-implemented  CBT  systems,  instructor  behavior  may  still  influence  achievement.  Rather 
than  trying  to  design  a  CBT  system  which  does  away  with  the  instructor  (or  to  design  a 
system  which  essentially  ignores  the  instructor),  CBT  developers  should  try  to  find  ways  in 
which  to  use  instructor  presence  to  maximize  achievement. 
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